Sparse Wide-Baseline Virtual View Generation

Given a number of images of a scene, Virtual View Generation (VVG) is the process of generating the images that would have been seen from viewpoints other than the one of the input images. As shown in Fig. 1, the movie industry already uses VVG for visual effect generation in films such as “The Matrix.” It could also allow a director to modify a scene by changing the viewpoint without the trouble and expense of reshooting it. Similarly, it is a necessary component of free viewpoint tv, which would allow a spectator to watch a show from any desired viewpoint. In many of today’s VVG systems, the approach is to shoot the scene using very many cameras and generate the video with a predefined trajectory that cannot depart much from the input camera array positions. Image synthesis mostly involves interpolation or re-sampling [7, 5]. In this thesis, we will develop an approach to VVG using a more limited number of cameras than what is the norm today. This will make VVG much more practical and flexible since it will require less effort to synchronize and calibrate and it will become possible to apply VVG to a more set of applications like surveillance systems where it is impractical and expensive to install a large number of cameras. However, achieving this will involve solving a number of severe difficulties. Since we do not have a dense sampling of the imaging space, interpolation and resampling based methods are not suitable. Therefore, additional cues like depth must be included to the formulation. Once the depth is known, it is not very difficult to synthesize new images. However, our case involves dense wide-baseline matching and handling of large occlusions resulting from the regions where only one camera is able to capture. These are problematic issues because: